Comparative Usability Study of Two Space Logistics Analysis Tools

Future space exploration missions and campaigns will require sophisticated tools to help plan and analyze logistics. To encourage their use, space logistics tools must be usable: a design concept encompassing terms such as efficiency, effectiveness, and satisfaction. This paper presents a usability study of two such tools: SpaceNet, a discrete event simulation tool and a comparable spreadsheet-based tool. The study follows a randomized orthogonal design having within-subjects evaluation of the two tools with 12 volunteer subjects (eight subjects with space backgrounds, four without). Each subject completed two sessions of testing, each with a 30-45 minute tutorial and a two-part space exploration scenario. The first part tests the creation a model to verify a simple uncrewed mission to lunar orbit. The second part tests the evaluation of an existing model to improve the effectiveness of a crewed mission to the lunar surface. The subjects completed a questionnaire after each session and a semi-structured interview following the second session. The study results indicate that the SpaceNet tool is more efficient for portions of the model creation task including modeling multi-burn transports and the spreadsheet tool is more effective for the model evaluation task. Qualitative evaluation indicates subjects liked the graphical nature and error-detection of the SpaceNet tool, but felt it took too long to edit information and appeared as a “black box.” Subjects liked the ability to view the entire model state within the spreadsheet tool, however were concerned with limited dynamic state feedback and underlying modeling assumptions. Future tools should combine the best features, including allowing modification of the entire model from a single interface, providing visibility of underlying logic, and integrated graphical and error-checking feedback.